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ABSTRACT 

This paper examines services being received by 
secondary students classified as seriously emotionally disturbed; 
school policies concerning their education; and the links between 
services, policies, and school performance. Data from a subset of 782 
youth with serious emotional disturbances from the National 
Longitudinal Transition Study (NLTS) were analyzed. This report 
describes the characteristics of the study population; the 
philosophies, policies, and practices of the schools they attended; 
and the services available in the schools. The report next examines 
these students 1 receipt of services and their secondary school 
performance and outcomes as measured by absenteeism, grades and 
course failure, retention in grade, performance on minimum competency 
tests, and mode of school leaving. Finally, multivariate models are 
used to examine the link between school policies, service receipt, 
and absenteeism and course failure. Results indicated that fewer than 
half of the students had received counsel ing/therapy from any source, 
and only one third'had received such services from their schools. 
Fifteen percent had received tutoring services. Very little of ths 
variance in school performance was explained by the models, though 
the models suggest that tutoring and personal counseling/therapy may 
help improve student outcomes . Appendices provide background 
information on the NLTS sample and a listing of 21 reports and papers 
based on the NLTS. Contains 41 references. (DB) 
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Federal law (formerly PL 94-142, currently PL 101-476) mandates that students 
who have disabilities that affect their ability to learn receive special education and/or 
related services. A student's educational program and related or support services can 
be important influences on his or her secondary school performance, along with 
individual and family characteristics (see Figure 1). However, recent findings 
concerning youth classified as seriously emotionally disturbed (SED) suggest that the 
educational system is not fully meeting their needs. For example, Kaufman et at. 
(1987) found that more than two-thirds of 9- to 17-year-old SED students performed 
below grade level. Furthermore, the Twelfth Annual Report of Special Education 
Programs to Congress (1990) indicates that 40% of secondary school exiters classified 
as SED dropped out of school. These discouraging findings suggest the need to 
examine what services are being received by students classified as seriously 
emotionally disturbed, school policies concerning their education, and the links between 
services, policies, and school performance. 

This paper conducts such an examination using data for 782 youth classified by 
their school districts as seriously emotionally disturbed. These youth constitute a subset 
of the National Longitudinal Transition Study (NLTS) sample. The NLTS is an ongoing 
study being conducted by SRI International under contract to the Office of Special 
Education Programs (OSEP) of the U.S. Department of Education. As part of that 
study, data were collected in the summer of 1987 for a nationally representative sample 
of more than 8,000 young people in all 1 1 federal special education disability 
categories who were between the ages of 13 and 21 and were enrolled as secondary 
special education students in the fall of the 1985-86 school year. Data were collected 
by telephone or in-person interviews with parents, from school records, and from a 
survey of educators in schools attended by study participants. (Appendix A has a more 
detailed description of data collection, data weighting and analyses. Appendix 6 lists 
other products available from the NLTS, including full reports on sampling and data 
collection methods.) 

This report describes ihe characteristics of secondary school students classified as 
SED, the philosophies, policies, and practices of the schools they attended, and the 
services available in them. The report then examines these students' receipt of 
services and their secondary school performance and outcomes as measured by 
absenteeism, grades and course failure, retention in grade, performance on minimum 
competency tests, and mode of school leaving. Finally, multivariate models are used to 
examine the link between school policies, service receipt, and absenteeism and course 
failure. 
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Characteristics of 1985-86 secondary school students classified as seriously 
emotionally disturbed 

It is estimated that there are more than 150,000 students in secondary schools 
classified as "seriously emotionally disturbed." 1 According to the federal government, 
this disability is defined as: 

Exhibition of behavior disorders over a long period of time that adversely affect 
educational performance; this includes an inability to learn that cannot be 
explained by intellectual, sensory, or health factors; an inability to build or 
maintain satisfactory interpersonal relationships with peers and teachers; 
inappropriate types of behaviors or feelings under normal circumstances; a 
general pervasive mood of unhappiness or depression; or a tendency to 
develop physical symptoms or fears associated with personal or school 
problems. 

Thus, youth with a variety of types of problems are included. Some of them suffer from 
neurobiological disorders, such as childhood schizophrenia or depressive disorder, 
while others have problems stemming from difficult family situations, such as abusive 
parents. 

Compared with secondary school students in the general population, students 
classified as SED are more likely to be male, black, from households whose heads 
have low levels of education, and from single-parent households. As shown in Table 1, 
about 75% of 1985-86 secondary school students classified as seriously emotionally 
disturbed were male; 2 67% were white, 25% were black, 6% were Hispanic, and 2% 
were of other ethnic backgrounds. In contrast, in the general population, about half of 
students were of each gender, 73% of students were white, 14% were black, 6% were 
Hispanic, and 7% were of other ethnic backgrounds. 3 About 33% of students 
classified as SED came from households whose head had fewer than 12 years of 
education, and 44% came from single-parent families. Strictly comparable data are not 
available for youth from the general population; however, data for 12- to 17-year-olds in 
the general population who were living with at least one parent indicate that about 22% 
of them came from households whose heads had fewer than 12 years of education, 
and 26% of them came from single-parent households (U.S. Bureau of the Census, 
1988). 



1 Based on unpublished OSEP data. 

2 Descriptive data are weighted so that findings generalize nationally to secondary school 
students classified as SED. NTs in tables refer to actual sample size used in each analysis. 

3 Data for the general population of youth come from the 1979-1983 National Longitudinal 
Survey of Youth (Center for Human Resource Research, 1990) and represent 13- to 21 -year- 
old secondary school students. 



Table 1 



Characteristics of 1985-86 Secondary School Students 
Classified as Seriously Emotionally Disturbed 



Percent Standard 

of Students 4 Error N 5 

Gender 782 

Male 76.6 (2.2) 

Female 23.4 (2.2) 

Ethnic background 644 

White 67.2 (2.6) 

Black 25.0 (2.4) 

Hispanic 6.0 (1.3) 

Other 1.7. (0.7) 

Head of household of origin 

has less than 1 2 years education 33.4 (2.4) 783 

Single-parent household of origin 44.3 (2.9) 598 

Age at which parents reported 

youth first experienced difficulty 546 

Before 1 year 8.9 (1.7) 

1-4 years 11.3 (1.9) 

5-8 years 41.4 (3.0) 

9-12 years 22.1 (2.5) 

13+ years 16.2 (2.2) 

Additional disability classification 779 

Any additional disability 24.3 (2.2) 

Learning disabled 22.7 (2.1) 

Mentally retarded 112 (1.6) 

Mean IQ score: 86.4 (1.1) 427 



4 Unless otherwise indicated, percentages of students in tables are weighted to represent all 13- 
to 21 -year-old secondary school students classified as SED in the United States in the 1985- 
86 school year. 

5 NTs shown in tables are the actual sample sizes used in analyses. 



According to parent reports, it was most common for youth classified as SED (who 
were 15 to 23 years old when data were measured) to have begun having disability- 
related difficulties when they were between 5 and 8 years old, during their first years in 
school. About 40% of youth began having difficulties during these years. About 10% 
began to have problems before they reached their first birthday, and another 10% 
began to have problems between 1 and 4 years of age. About 16% began to 
experience problems after age 13. 

Many students in special education have more than one disability (Marder & Cox, 
1991). The students represented in this report were classified as having a primary 
disability of serious emotional disturbance. However, about 1/4 of students classified 
as seriously emotionally disturbed also had a secondary disability classification. The 
most frequently occurring secondary classifications were learning disabled (23%) and 
mentally retarded (8%). 

The mean IQ score for youth classified as SED was 86. 6 This relatively low mean 
score is consistent with findings of past research, which also found youth with scores 
below 70 (who would be classified as mentally retarded in most states) included among 
youth classified as SED (e.g., Kaufman et al., 1987). 

The sample used in this report includes youth from 15 to 23 years old (Table 2). 
When weighted (as for descriptive analyses in this report), 37% of youth were 15 ano 
16 years old, and about the same percentage were 17 and 18 years old. About 20% of 
the weighted sample were 19 or 20 years old, and about 4% were over 21 . 

What were the philosophies and practices of the secondary schools attended by 
students classified as SED? 

This section reviews the general philosophy, policies, and practices concerning 
mainstreaming, grading, and minimum competency tests of secondary schools 
attended by students classified as seriously emotionally disturbed. Because the great 
majority (87%) of such students attended regular secondary schools, as opposed to 
schools that served only special education students, the remainder of this report 
focuses only on students in regular schools. 

School Philosophy 

The primary goal of secondary schooling for students in general has changed 
various times during this century (Grubb, 1989), and at present there is no consensus 



6 The most frequently used test among this sample was the Wechsler with a mean of 100 and a 
standard deviation of 15. Although IQ scores were available only for 75% of youth in the 
sample classified as SED, findings based on investigation of the correlation of IQ scores with 
other measures of mental ability suggest that there is no bias in the scores. 
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Table 2 



Age of Youth Classified as Seriously Emotionally Disturbed 
in the Weighted NLTS Sample 



Percent 



Standard 
Error 



N 



Percentage of students 
whose age was 



779 



15-16 
17-18 
19-20 
21 + 



36.9 
39.0 
20.3 
3.8 



(2.4) 
(2.5) 
(2.0) 
(10) 



as to what the primary function of secondary schools should be. There is even less 
consensus concerning what the main role of the school should be for students with 
disabilities. Should schools limit their focus to teaching academic skills to students 
with disabilities? Or should their main mission be to teach learning handicapped 
students other skills, such as job skills and independent living skills? Understanding 
the goal of education for students with disabilities is so important that it has been posed 
as the number 1 question by the National Association » f State Boards of Education for 
consideration by its members (Roach, 1991). 

The NLTS asked schools what they saw as their primary function in serving learning 
handicapped students. Schools attended by about half of secondary students 
classified as SED (52%) indicated that their primary function in serving learning 
handicapped students was to teach academic skills (Table 3), whereas schools 
attended by about 1/3 of students classified as SED saw their primary function as 
teaching independent living skills. Only 5% of students attended schools who saw their 
main purpose as training students for employment. 

Mainstreaming 

Individualization of instructional programs is central to PL 94-142; such 
individualization is seen as necessary to compensate for or circumvent the educational 
obstacles presented by disabilities. Special education classes often are smaller and 
are more likely to have instructional aides present in addition to the teacher, "clearly 
offering the opportunity for more individualized and small group instruction' 1 (Singer et 
al., 1986). 



a 
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Table 3 



Philosophy of Regular Secondary Schools Attended by 
Students Classified as Seriously Emotionally Disturbed 



Percent of 
Students 
Attending 
Schools 



Standard 
Error 



The primary function of school to the 
learning handicapped is to: 

Teach academic skills 
Teach independent living skills 
Train students for employment 



51.7 
34.3 
5.2 



(3.4) 
(3.3) 
(1.5) 



439 



On the other hand, regular education classes offer students with disabilities very 
important opportunities to form friendships and model the behavior of nondisabled 
students and more challenging curricula (Kaufman, 1991). Equity concerns also 
encourage integration so that all students who can benefit from regular education 
instruction have the opportunity to do so. Accordingly, the maximum appropriate 
integration of special education students with the general student population is the 
specific intent of the "least restrictive environment" provision of PL 94-142, and since 
the passage of PL 101-476 the Department of Education appears to be especially 
encouraging mainstreaming of students classified as SED (Kaufman, 1991). 

Consistent with the legislative push for mainstreaming, in the 1985-86 and 1986-87 
school years, most students classified as SED spent the majority of their time in regular 
education classes. Twenty percent spent their full school day in regular education 
classes (Table 4), and similar percentages spent between 75% and 99% and between 
50% and 74% of their time mainstreamed, respectively. Twenty-five percent of 
students were mainstreamed between 1% and 50%, and only 13% spent all their time 
in special classes. 

Students were slightly more likely to be mainstreamed in nonacademic courses than 
in academic courses. In 1985-86 and 1986-87, more than 80% of students who took 
nonacademic courses were mainstreamed for at least one of them, whereas the 
comparable figure for academic courses was 70% (p<.01). Among students taking 
vocational courses, about 77% were mainstreamed for at least one of them. 

The extent to which students can succeed in regular education classes may depend 
a great deal on whether or not they are expected to keep up with other students without 
special help. For many special education students, the expectation of keeping up with 
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Table 4 

Policies and Practices Regarding Mainstreaming 
of Regular Secondary Schools Attended by Students 
Classified as SED 



Percentage 

of Standard 

Students Error N 

Percent of time in regular education: 447 

0 13.0 (2.2) 

1-24 10.4 (2.0) 

25-49 14.7 (2.4) 

50-74 21.0 (2.7) 

75-99 20.7 (2.7) 

100 20.0 (2.6) 

Of students who took a type of course, those who 

took at least one of them in regular education classes: 

Academic* 70.1 (3.1) 426 

Vocational** 76.9 (3.3) 326 

Nonacademic*** 82.7 (3.2) 381 

Schools expect mainstreamed students to keep 

up with rest of class without special help 39.9 (3.4) 442 

Services available to regular education 
teachers when special education students 

are mainstreamed in their class 443 
Consultation services by soecial education or other 

staff 96.9 (1.2) 

Special materials to use with mainstreamed students 48.2 (3.4) 

Inservice training on needs of mainstreamed students 39.7 (3.3) 

Human aides 26.5 (3.0) 

Smaller classes 12.5 (2.3) 

None of the above 0.3 (0.4) 



* Academic courses include English/language arts, mathematics, science, social 

science, and foreign language. 
** Vocational courses include home economics, courses in prevocational skills, and 

occupational^ specific courses. 
*** Nonacademic academic courses include physical education, music, art, drivers' 

education, etc. 

11 
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the rest of a regular class without special help may result in a great deal of frustration, 
poor grades, and failure. Under these circumstances, students may be reluctant to 
participate in regular classes, and schools may be reluctant to place them in regular 
classes. The NLTS asked schools, "When your school mainstreams special education 
students, are they usually expected to keep up with the rest of the class without special 
help?" In the 1985-86 and 1986-87 school years, 40% of secondary students classified 
as SED attended schools that answered "yes" to this question, and the expectation was 
associated with the amount of time students were mainstreamed. 

Despite this stated policy, virtually all schools made some type of training, service, 
or materials available to regular education teachers when they had special education 
students in their classes. In schools attended by almost all students classified as SED 
(97%), special education staff or other staff were available for consultations with 
teachers of mainstreamed students. Other types of materials and services were far 
less common, however. Special materials to use with mainstreamed students were 
available in the schools attended by about half of the students. Schools attended by 
about 40% of the students had inservice training on the needs of mainstreamed 
students for teachers. Measures to reduce the teacher: student ratio in classrooms with 
mainstreamed students were rarer. Schools attended by only 1/4 of students classified 
as SED had human aides in classrooms with mainstreamed students, and schools 
attended by only 13% reduced ciass sizes when students were mainstreamed. 

Grades 

Grades are the third area of a school's policy that can have powerful effects on 
students' experiences. Course grades provide students with often-powerfu! messages 
about not only their academic status but their more basic abilities, their standing in their 
classes, and their value as students. These messages often are more personal than 
ratings provided by standardized test scores, for example, because they reflect the 
assessments made by specific teachers with whom students have individual 
relationships. Over the course of an entire school career, grade performance helps 
shape students' images of themselves as learners and of their competence to perform 
academic tasks (Bloom, 1976; Finn, 1989). Given the profound effects grades can 
have on students' self image, schools are faced with the choice of whether the work of 
students with disabilities should be graded at all and, if so, whether it should be held to 
the same standards as other students. 

In 1986-87, more than 90% of students classified as SED in regular secondary 
schools received grades (Table 5), and more than half of them (61%) attended schools 
that indicated that the same grading standards were applied to the work of special 
education students in regular classes as to the work of other students. Not surprisingly, 
for work done in special classes, different grading standards were more likely to be 
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used. Nevertheless, about 1/4 of students attended schools that graded work done 
special classes by the same standards as work done by non-special education 
students. 

Table 5 

Policies and Practices Regarding 
Grading and Minimum Competency Tests 
of Regular Secondary Schools Attended by 
Students Classified as Seriously Emotionally Disturbed 



Grading 

Received grades 

Students who attended schools in which: 
Special ed students in regular classes 
were graded using the same standards 
as other students 

Special ed students in special classes 
were graded using the same standards 
as other students 



Minimum competency tests 

Exempted from requirement 



Percentage Standard 

of Students Error N 

91.3 (1.8) 502 

61.5 (6.8) 117 

23.6 (5.9) 117 

22.2 (3.6) 273 

58 

(10.4) 
(10.4) 
(6.5) 
(6.9) 



Of students who took MCT, those in schools 
where: 

Same version and standards for completion 

of test were used for special ed 

students as for as other students 69.9 
Special ed students given assistance 

in taking test 44.4 
Special ed students given modified 

version of test 10.8 
Different standards for successful 

completion of test 12.2 
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Minimum competency tests 

In recent years, because of concern that substantial numbers of students were 
exiting secondary school without a basic body of knowledge, many states have 
instituted minimum competency tests (MCTs). In general, the purpose of such tests is 
two-fold: first, to guarantee that students have acquired certain basic knowledge 
before they are promoted to the next grade or are graduated, and, secondly, to 
enhance students' academic skills (Serow, Davies, & Parramore, 1982). However, their 
use has raised some concern in the special education community that MCTs could act 
as barriers to graduation for students with disabilities. These concerns are increased 
by such findings as reported by Allen, Rawlings, and Schildroth (1989) that deaf 
students were less likely to graduate with diplomas and more likely to receive 
certificates in states where minimum competency tests were required to obtain a 
diploma. 

States, districts, and/or schools have a range of options regarding MCTs for special 
education students, from exempting them from the tests completely to requiring them to 
take and pass the same tests as other students with no assistance. About 1 in 4 
students classified as SED attended schools that administered MCTs but exempted 
special education students from them. Of those who were subject to the tests, 70% 
attended schools that required special education students to take the same version of 
the test and pass according to the same standards as other students. However, 
schools attended by almost half of these students (55%) reported that special 
education students were given some assistance in taking the test. 7 Schools attended 
by 11% of students classified as SED who were subject to MCTs indicated that special 
education students were given modified versions of the tests, and schools attended by 
12% reported that standards for successful completion of the tests differed for special 
education students and other students. 

What disability-related services were available In secondary schools attended by 
students classified as SED? How much coordination was there between schools 
and other agencies? 

By definition, special education students need some type of special help to benefit 
from the educational process. For some students, the help comes in the form of 
special classes; however, for others, particularly those who are mainstreamed, other 
types of services are important. The nature of their disability suggests that many 
students classified as seriously emotionally disturbed would benefit from psychological 
counseling or therapy. In addition, although they may not help solve the root problems 

7 The nature of the assistance was not reported; therefore, the assistance could be relevant or 
not for students classified as SED. For example, by "special assistance" a school might refer 
only to interpreters and/or readers for students with sensory impairments, which would not 
affect students classified as SED. On the other hand, "special assistance" might include extra 
time or other special conditions for students classified as SED. 
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of these students, tutors and/or human aides can play important roles in helping 
students to master schoolwork. Furthermore, for those students whose emotional 
disturbances have external causes, such as severely dysfunctional families, social 
workers can be at least as important as psychological counseling. To what extent are 
these types of services available in schools attended by youth classified as SED? 

Although the school is the logical provider of some types of services (e.g., tutoring), 
students may receive psychological and social services from other sources. Indeed, 
there have been allegations that families are pressured by schools to obtain 
psychological services for students who need them from outside the schools (Knitzer, 
Steinberg, & Fleisch, 1990). To the extent that students receive services from sources 
other than schools, coordination between schools and the service providers is 
important (Kaufman, 1991). With coordination, students' chances of receiving the 
services they need are enhanced, and the possibility of various agencies and service 
providers working at cross purposes is reduced. In this section, we examine the extent 
to which services were available in regular secondary schools attended by youth 
classified as SED, and to what extent schools coordinated services with other 
agencies. 

In-School Services 

The NLTS asked schools whether they usually made available counseling or 
psychotherapy, social work, and tutors or human aides to special education students 
who needed them. Secondary schools attended by most students classified as SED 
indicated that they provided all three types of services; almost 70% provided counseling 
or psychotherapy for disability-related problems (Table 6), close to 80% provided social 
work, and about 85% provided tutors or human aides. While these percentages may 
seem high, the "glass" seems far too empty when one considers the fact that more than 
30% of students classified as SED attended schools that did not provide counseling or 
psychotherapy, more than 20% had no access to social workers through their school, 
and almost 15% had no access to human aides or tutors. 

Past research has painted a discouraging picture of the postschool outcomes for 
special education students in terms of employment, enrollment in postsecondary 
schools, and independent living (e.g., Edgar, 1987; Hasazi, Gordon, & Roe, 1985; 
Mithaug, Horiuchi, & Fanning, 1985; Sitlington, Frank, & Cooper, 1989). Given these 
poor outcomes, in recent years there has been considerable attention to the need for 
programs specifically designed to assist special education students in making the 
transition from secondary school to adult life. Despite this emphasis, in the 1986-87 
school year transition programs were available in schools attended by only 14% of 
students classified as SED. It is likely that this picture will change drastically with the 
passage of the Individuals with Disabilities Education Act (PL 101-476), which 
mandates that schools establish transition programs. 
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Table 6 



Services Available in Regular Secondary Schools Attended by 
Students Classified as Seriously Emotionally Disturbed 



Percentage 
of Students 

Attending Standard 
Schools Error N 



School usually makes available to 
special ed students: 

Counseling or psychotherapy for 

disability-related problems 
Life skills/occupational therapy 
Social work 
Human aides or tutors 

Special program to help special education 
students with transition 



68.9 


(3.3) 


414 


88.6 


(2.1) 


431 


77.8 


(2.9) 


420 


86.3 


(2.4) 


440 


13.8 


(2.5) 


404 



Coordination with Other Service Agencies 

As stated earlier, when outside agencies provide services to students, coordination 
of services among those agencies and the school is important. Typically, coordination 
of services can happen by schools being in direct contact with other agencies regarding 
the services students are receiving or by students having case managers who oversee 
and coordinate all the services and programs in which a student participates. 

Schools attended by most students classified as SED appear to have relied more 
on case managers for coordination than on direct contacts with other agencies. 
Althougn schools attended by almost all students reported that they had some contact 
with mental health and social service agencies, the frequency of contact was typically 
very low, with schools attended by the vast majority of students indicating that they 
were in contact with these agencies no more than "a couple times a year." Schools 
attended by only about 1/4 of students reported being in contact with these types of 
agencies at least monthly (Table 7). In contrast, schools attended by about 87% of 
secondary students classified as SED indicated that "special education students 
typically had case managers or a person assigned to them who is responsible for 
coordinating the services they receive." 
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Given the high percentage of schools that reported that students had case 
managers, one might imagine that all schools either had frequent contact with agencies 
or assigned case managers; however, schools attended by 1 in 10 students classified 
as SED had neither case managers nor frequent contacts with mental health or social 
service agencies. 

Table 7 

Coordination of Services with Outside Agencies 
in Secondary Schools Attended by Students Classified as SED 

Percentage 
of Students 



Attending Standard 

Schools Error N 

Special education students have 

case managers 86.6 (2.3) 446 

Frequency of contacts with outside agencies 

Mental health agencies 429 

A couple of times per school year or less 87.6 (2.8) 

Monthly 12.4 (2.8) 

• Weekly 9.0 (2.0) 

Social service agencies 424 

A couple times per school year or less 72.8 (3. 1 ) 

Monthly 18.4 (2.7) 

Weekly 8.8 (2.0) 

Coordination of services by case 
managers or frequent direct contact 
with other mental health or social 

service agencies 365 

Neither case manager nor frequent contact 10.0 (2.2) 

Case managers only 63.7 (3.6) 

Frequent contact with agency 3.9 (1 .4) 

Both 22.4 (3.1) 
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What services were secondary school students classified as SED receiving? 

Although, as we saw in the last section, schools indicated that they made services 
available to students, how many secondary school students classified as SED were 
actually receiving services, what types of services were they receiving, and how much 
of a given service did they receive? The answers to these questions come both from 
parents 1 and schools' reports of youths' receipt of services and from abstracts of school 
records. The three types of services considered in this report are personal counseling 
or therapy, help fro .71 tutors or readers, and occupational therapy or life skills training. 

Despite the importance of psychological counseling or therapy for students 
classified as SED, there is no mandate for the provision of such services. In fact, the 
percentage of students that actually received such services is rather low; in the 12- 
month period ending summer 1987, fewer than half (43%) of students with this 
classification in regular secondary schools received such services (Table 8). Of these, 
most students received services from their schools. Nevertheless, in all, only about 1/3 
of students classified as SED in regular secondary schools received personal 
counseling/therapy from their schools. About 7% received counseling/therapy from 
private therapists, and 6% from other service agencies. 

Help from tutors or readers and life skills training/occupational therapy were less 
common than personal counseling. Only 15% of students classified as SED received 
such help by tutors or readers, and about 20% received life skills training or 
occupational therapy. 8 These types of services were provided almost exclusively by 
schools. 

Looking at receipt of each service separately might lead one to think that almost all 
students were receiving some type of service. However, this was far from the case. In 
1986-87, although 38% of regular secondary school students classified as SED 
received one service and 17% of students received multiple services, 43% received no 
services at all. Among students who received any service, the most common pattern 
was to receive only one— personal counseling or therapy. 



What were the outcomes of secondary school students classified as SED? 

Secondary school performance and outcomes can be measured in a variety of 
ways. The importance of grades and minimum competency tests and the effects they 
may have on students have been discussed earlier. To these measures of 
performance, in this section I add absenteeism, course failure, retention in grade, and 



Life skills training/occupational therapy includes a wide variety of types of instruction, from 
grooming to cooking and housekeeping skills. See Appendix C in Wagner et al., 1991, for a 
description of the definition and operationalization of this and other variables used in this 
report. 
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Table 8 



Services Received in a 12-Month Period 
by 1986-87 Secondary School Students 9 
Classified as Seriously Emotional Disturbed 





percentage 


oianaara 






of students 


Error 




Counseling or therapy 10 








From any source 


43.0 


(3.3) 


452 


From secondary school 


33.8 


/O A \ 

(3.1) 


452 


Private therapist 


7.0 


(1.8) 


420 


Other service agency 


6.2 


(1.7) 


076 


Tutor or reader 






AC A 

4o1 


From any source 


15.0 


(2.4) 




From secondary school 


12.1 


(2.2) 




Occupational therapy/life skills training 






453 


From any source 


iy.4 


/o cz\ 
(2.6) 




From secondary school 


A O A 

18.1 


/o c\ 

(2.5) 




Types of services received 






449 


No services at all 


43.9 


(3.3) 




One service only 








Counseling /therapy 


OK Q 


(^.y) 




Tutor/reader 


o.z 






Occupational therapy/iife skills 


1 o 


(l.f) 




More than one service 








Counseling/therapy and tutor/reader 


5.4 


(1.5) 




Counseling/therapy and occupational 








therapy/life skills 


7.9 


(1.8) 




Counseling, tutor/reader,, and life skills 


3.4 


(1.2) 




Receipt of services and/or reduced 








student:teacher ratios in regular education 








classes:* 






371 


Neither ■ 


29.1 


(3.3) 




Reduced class size or human aides 


11.4 


(2.3) 




Some service 


46.0 


(3.7) 




Services and reduced class size or human aides . 13.6 


(2.5) 





9 Does not include students in special schools. 

10 Percents do not sum to 100 because some students received more than one type of service. 
* Excludes students with no regular education classes. 




the ultimate outcome of secondary school, which is whether youth drop out of 
secondary school. 11 



Absenteeism 

A minimum expectation for student performance is that students attend school; 
without participation in the educational process, its benefits are difficult to attain. 
Although students with disabilities may need to miss school for reasons related to their 
disabilities, such as illness or treatments, there also may be a voluntary component to 
some absenteeism. Peng, Fetters, and Kolstad (1981) report that 20% of 1980 high 
school seniors in the general population reported being absent from school more than 5 
days in the school year for reasons other than illness. 

High absenteeism, whether voluntary or involuntary, presents significant challenges 
to meeting academic standards. Past research has identified absenteeism as perhaps 
the single strongest predictor of academic failure and dropout decisions for students . 
with disabilities (Thornton et al. a 1987; Donahoe & Zigmond, 1990; Schellenberg, Frye, 
& Tomsic, 1988). Using NLTS data, Wagner (1991) also found absenteeism to be a 
powerful predictor of both of these for youth with disabilities as a group. 

According to students' school records collected by the NLTS, rates of absenteeism 
among secondary students classified as SED appear to be very high. 12 Fewer than 1/4 
of these youth were absent 5 days or less in their most recent school year (Table 9), 
and almost 1/3 were absent more than 20 days. These rates of absenteeism were 
much higher than for high school students in general, among whom 37% were absent 5 
or fewer days a year and 17% were absent more than 20 days. 



11 All measures of school performance discussed in this section were collected by the NLTS 
from abstracts of records of students' most recent year in school. Data for mode of school 
leaving come both from students' school records and from parent reports (see Wagner et al., 
1991 , for a description of the various components of Wave 1 of the NLTS and complete 
variable descriptions). 

12 Data for the number of days absent in the most recent school year were missing in 15% of the 
abstracts of records received from schools. No significant differences were found between 
those students for whom data were provided and those for whom data were missing on IQ 
scores or GPA. There was, however, a significantly greater absence of data for students in 
middle school (grade levels 7 or 8) than higher grades (23% missing vs. 11% to 13% missing, 
p<.01). Because younger students had somewhat lower rates of absenteeism, the 
underrepresentation of these students would slightly inflate overall absenteeism levels. 
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Table 9 



Seiected Outcomes of Regular Secondary School Students 
Classified as Seriously Emotionally Disturbed 
in Most Recent School Year 



Percentage 
of Students 



Standard 
Error 



N 



Days absent 

0-5 

6-10 

11-20 

21-30 

30+ 

Mean days absent: 16.7 



444 



23.4 
21.0 
26.2 
14.0 
15.4 



(3.1) 
(3.0) 
(3.2) 
(2.5) 
(2.6) 

(1.1) 



Grades 
GPA 

3.25 or higher 3.2 
2.75-3,24 8.4 

2.25-2.74 19.3 

1.75-2.24 22.3 

1.25-1.74 20.1 

< 1.25 27.0 

Mean GPA: 1.7 

Course failure 

Of students receiving grades 

percentage failing one or more course 43.9 

Of students who had graded regular education 

classes, percentage failing one or more 44.6 

Of students who had graded special education 

classes, percentage failing one or more 25.9 



(1.2) 
(1-8) 
(2.6) 
(2.7) 
(2.6) 
(2.9) 

(0.6) 



451 



451 



451 



(3.1) 504 
(3.5) 384 
(3.0) 413 



Retention in grade 

Of those who remained in school, percentage 
retained in grade 



16.1 



(3.2) 



245 
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Table 9 



Selected Outcomes of Regular Secondary School Students 
Classified as Seriously Emotional Disturbed 
in Most Recent School Year 

(concluded) 



Minimum Competency Tests 

Students subject to MCTs but exempted 
from the requirement 

Of students who took the test 
Passed the entire test 
Passed some of the test 
Failed the test 



Percentage 
of Students 



22.2 



36.6 
40.8 
22.7 



Standard 
Error 



3.6 



(5.0) 
(5.1) 
(4.3) 



N 

273 
189 



How exiters left school 

Graduated or certificated 
Dropped out 
Suspended or expelled 
Aged out 



232 



49.9 
44.0 
4.8 
1.2 



(4.8) 
(4.8) 
(2.1) 
(1.1) 



Grades 

The powerful role that grades play in shaping a student's experiences has been 
discussed earlier. According to their grade point averages, 13 most students classified 
as SED were performing below average. More than 1 in 4 students had GPAs of less 
than 1 .25, and another 1 in 5 had GPAs between 1 .25 and 1 .75. In other words, 
almost half of students classified as SED had below "C-" averages. Very high grades 
were quite uncommon; only 3% of students had GPAs of 3.2^ or higher, and 8% had 
GPAs between 2.75 and 3.24. 

Although overall grade point average summarizes a student's general grade 
performance, unless it is very low, it does not indicate whether a student failed a 
course. Secondary students may have had a C average or even higher and still have 
failed one or more courses, with a resulting loss of credits needed for graduation. 



13 Grade point average is calculated on a 4-point scale, with a grade of A assigned 4 points, B 
assigned 3 points, C assigned 2 points, D assigned 1 point, and failed courses assigned no 
value. Points are summed and divided by the total number of courses, including those failed. 
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NLTS data indicate that 44% of regular secondary school students classified as 
seriously emotionally disturbed failed at least one course in their most recent year of 
school. Most of these failing grades occurred in regular education classes. Forty-five 
percent of students who had graded regular education classes failed at least one of 
them. In contrast, 26% of students in graded special education classes failed at least 
one. 

These findings must be interpreted with extreme caution. Although course grades 
are often-used indicators of school performance, their limitations for research purposes 
are well known. Grade inflation makes comparisons of grades across time suspect. 
Variations in grading standards across schools and districts reveal that aggregated 
grade data often obscure more differences than they reveal even for students in the 
general population, in special education, a further difficulty in understanding school 
performance from course grades results from variations in the policies and standards 
used to grade the work of special education students. As shown in Table 5, not all 
special education students in regular schools received grades. Furthermore, some 
schools apply the same standards to special education students as to other students 
while other schools do not. For all these reasons, the grades of students with 
disabilities cannot be used for comparisons with the general population or any other 
fixed standard. Nevertheless, they are indicative of the messages students received 
about their competence to do the learning tasks expected of them. 

Retention in Grade 

A fundamental measure of school performance is meeting the expectations of 
performance for a given grade level and being promoted to the next grade level at the 
end of the year. Students who do not meet grade-level expectations repeat a grade, 
with the hope that further exposure to the required materials will help them master the 
skills and knowledge they failed to acquire on the first attempt. Although there has 
been some controversy over the academic benefits of nonpromotion (Center for Policy 
Research in Education, 1990; Gallup, 1989; Shepard & Smith, 1989; Marion & 
Coladarci, 1990), it appears to have strong emotional costs for students who feel 
punished and stigmatized (Bachman, Green, & Wirtanen, 1971; Byrnes, 1989; Shepard 
& Smith, 1989). Yamamoto (1980) also reports that children rate being retained in 
grade as an extremely stressful life event. 

The long-range prospects for those who experience nonpromotion are not good. 
Some have estimated that students who have repeated a grade are between 4 and 5 
times more likely to drop out of school than are students who did not repeat grades 
(Bachman et al., 1971; Raber, 1990). The effect of grade retention on dropout rates 
apparently is independent of student achievement (Grissom & Shepard, 1989). 

?3 



ERIC 



20 



The retention rate 14 for a single school year of students classified as seriously 
emotionally disturbed was 16%. This rate is considerably higher than estimates for the 
general population of students, which range from 4% to 9% (Shepard & Smith, 1989), 
and is the highest of secondary school students with any disability classification — 9 
percentage points higher than that of students classified as learning disabled (p<.05; 
Wagner, 1991). 

In addition to low grades, failure of minimum competency tests may lead to students 
being retained in grade. Among students classified as SED, just over 1/3 of students 
who took MCTs passed the entire test, and another 40% passed part of the test. 
Almost 1 in 4 students failed the test. Although not strictly comparable, data reported 
on students in the general population in Louisiana suggest that their rates of passing 
MCTs were similar to those of youth classified as SED. Brooks and Pittman (1990) 
report that 69% of 10th graders in that state passed the language arts portion of the 
MCT and 55% passed the mathematics portion. 

s 

Mode of School Leaving 

Whether or not a person graduates from secondary school has been found to have 
serious consequences. For example, in the general population, secondary school 
graduates have significantly higher employment rates and wages than nongraduates 
(Mincer, 1990; Murphy & Welch, 1990). Furthermore, lack of a high school diploma 
considerably limits an individual's postsecondary education options. Among youth with 
disabilities, the NLTS has found dropping out of school to be associated with poorer 
employment outcomes, lower rates of postsecondary attendance, and higher arrest 
rates in the first 5 years after leaving secondary school (Wagner et al., 1992). Yet, 
despite its high costs, dropping out often follows poor performance in secondary 
school. 

Given their poor showing on all other measures of secondary school performance, it 
is not surprising that a high percentage of youth classified as seriously emotionally 
disturbed dropped out of secondary school. Among youth with this classification who 
exited secondary school in the 1985-86 or 1986-87 school years, almost 45% dropped 



14 The NLTS measured the extent of grade retention for students with disabilities during their 
most recent year in secondary school. The rate of retention was calculated for all students 
who were assigned to a grade level and who remained in secondary school. Students not 
assigned to a grade level were eliminated from the analysis because they generally do not 
advance from grade to grade in the same manner as other students. Students who dropped 
out, withdrew, or moved during a school year were also excluded from the analysis because it 
is unknown whether they would have been promoted had they remained in school. The 
retention rate, therefore, is the percentage of students assigned to a grade level and still in 
school who were retained in grade, rather than being promoted to a grade, at the end of the 
school year. 



224 



out. Furthermore, an additional 5% of youth did not return to school after they were 
suspended or expelled. Thus, only half of all youth classified as seriously emotionally 
disturbed who left secondary school did so by graduating. 

How did services and other factors relate to the performance of secondary school 
students classified as SED? 

As stated in the introduction, the goal of school policies and services is to improve 
the schooi performance and outcomes of students. Thus far, this report has presented 
a description of secondary school students classified as SED, reviewed the 
philosophies and policies of regular secondary schools attended by students classified 
as SED, and has described the secondary school performance and outcomes of this 
group of students. Yet the questions remain: To what extent do personal 
characteristics of youth classified as SED influence their performance and outcomes in 
secondary school? Tc what extent do schools have an impact? 

To explore these questions, I estimated multivariate models of two measures of 
school performance: absenteeism and course failure. (Unfortunately, a model of 
dropping out could not be estimated because of the small number of dropouts for 
whom data on independent variables were available.) Absenteeism is measured as 
days absent in the most recent school year, course failure is measured as having failed 
at least one course in the most recent school year. Both models were estimated using 
data for all youth in the sample who were students in regular secondary schools in the 
1 986-87 school year. 15 

With these exclusions, and because of missing data, the samples for the models 
are considerably reduced from the original sample of 779 described in Table 1 Means 
and standard deviations of the samples used in the models are presented in Appendix 
C. As in Table 1, the descriptive statistics presented in Appendix C are weighted; 
however, models were estimated using unweighted data. 16 

Despite the dichotomous nature of two of the dependent variables, I estimated all 
models using linear regression. Although logistic regression is generally preferable for 
estimating models with dichotomous dependent variables, after estimating models with 
both methods and confirming that results were substantially the same, I chose to 
present results from the linear regressions because of their ease of interpretation and 
familiarity to most readers. The coefficients presented for the models with dichotomous 
dependent variables indicate the effect of each covariate on the probability of the 
dependent variable being whatever is represented by "1," when all other variables in 



15 Youth who left school before the 1986-87 school year were excluded from the analyses so 
that variables regarding service receipt (which covered only the 12-month period preceding 
the interview) would relate to service receipt while in secondary school. 

16 Given correctly specified models, weighting would not change the results. 
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the model are held constant. To illustrate, consider the dependent variable "failing a 
class." A coefficient of .1 for a covariate would mean that for each unit change in the 
covariate, a person would have a 10% greater probability of having failed a class. 



I estimated models with various specifications, testing the hypotheses implied by 
the conceptual model presented in the introduction — that both individual-level and 
school-level factors are associated with both dependent variables. However, because 
of the small sample size, models could contain only a limited number of variables. 
Therefore, when a variable was not of primary interest, and I consistently found no 
statistically significant effects between it and either dependent variable regardless of 
the other variables in the model, and the coefficients of other variables did not change 
with its exclusion, I excluded the variable from the final models presented in Table 10. 

This was the case for most demographic variables traditionally found to be 
associated with secondary school performance and outcomes. Specifically, I found no 
'statistically significant associations of gender, ethnic background, coming from a 
household whose head had less than 12 years of education, or coming from a single- 
parent household had any significant association with number of days absent or with 
probability of failing at least one course for students classified as seriously emotionally 
disturbed who attended regular secondary schools. 

The models presented in Table 10 show the association of students' absenteeism 
and course failure with IQ score; having received in the preceding 12 months personal 
counseling or therapy; help from a tutor, reader, or interpreter; or occupational therapy 
or life skills classes; the school's philosophy concerning its primary function for 
learning-handicapped students; whether special education students who were 
mainstreamed were expected to keep up without special help; and the percentage of 
time a student spent in regular education classes. The models also test the notion that 
absenteeism is associated with course failure. 

Absenteeism 

We can say. very little concerning absenteeism because, regardless of the 
specification of the model, very little of the variance in days absent was explained. 
Indeed, the adjusted r 2 of .04 for the model in Table 10 was the largest amount of 
variance explained with any specification of the model estimated. Despite the very low 
amount of variance explained, however, the model suggests that receiving tutoring 
services, being in a school that expects mainstreamed students to keep up without 
special help, and spending more time in regular education classes may decrease 
absenteeism. 
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Table 10 



Factors Related to Absenteeism and Course Failure 
of Students in Regular Secondary Schools 
Classified as Seriously Emotionally Disturbed 



Unstandardized Regression Coefficient 
Days 



Youth characteristics 

Measured IQ 

Services received in . 
preceding 12 months 

Personal counseling/therapy 

Tutoring/reading 

Life skills/occupational therapy 

School philosophy and policies 

Primary function: teach academics 
Mainstreamed students expected to 
keep up without special help 

In most recent school year 

Each 10% of time in regular ed 
Number of days absent 



Absent 17 

0.11 



-2.12 

-6.44* 

-1.28 



-1.33 
-4.82- 

0.79- 



Course 
Failure 16 

0.00 



-0.15* 

-0.06 

-0.04 



-0.15* 
0.00 



-0.01 
0.01- 



N 

Adjusted r 2 



190 
.04 



190 

.09 



+ p<.10; *p<.05; -p<.01; —p<.001. 



17 Includes youth who were in school and out of school less than 1 year. 
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Tutoring services may help lower absenteeism by helping students have a better 
grasp of their work, or at least of how to approach it. Even if performance on actual 
school work is not immediately affected, improved knowledge of how to tackle 
assignments would almost certainly decrease feelings of frustration that could make a 
student reluctant to attend school. 

Schools expecting mainstreamed students to keep up with the rest of the class and 
percent of time in regular education classes may be associated with lower absenteeism 
for several reasons. On one hand, once receipt (or nonrece pt) of tutoring is held 
constant, high expectations for students and the experiences of challenging curricula 
and interactions with other students in regular classes may make school a more 
rewarding and interesting experience for them, thus making them less reluctant to 
attend. On the other hand, unmeasured differences between students who attended 
schools where they were expected to keep up and students who attended other 
schools, and between students with more and less time in regular education classes 
may well account for some of the associations. To the extent that students in school 
with "sink or swim" policies and students who were mainstreamed more were less 
severely disabled, these variables may be proxying for differences in severity of 
disability. 

Failing a course 

Models of course failure also were able to explain little of the variance (r 2 =.09). 
Interestingly, none of the variables that are associated with absenteeism have direct 
relationships with course failure. Furthermore, a student's IQ also is not associated 
with course failure. However, as expected, absenteeism appears to have a strong 
association with failing a course. For each day absent, students' probability of failing a 
course increased by 1%. On the other hand, although neither tutoring/reading nor 
occupational therapy/life skills appears to be related to course failure, having had 
personal counseling or therapy in the preceding 12 months appears to reduce the 
probability of course failure. 

A school's philosophy also appears to be directly associated with course failure. 
Students in schools that saw their primary function for learning handicapped students 
as teaching academic skills were more likely to fail courses than students in schools 
that saw their main mission as teaching independent living skills or training students for 
competitive employment. Precisely how schools' philosophies translate into what 
students experienced, affecting their probability of course failure, is unclear. Among 
various possibilities are that schools that saw their main mission as teaching academics 
had more rigorous standards than other schools, or had more limited curricula that were 
less well suited to the abilities and interests of many students classified as SED. 
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Summary and Conclusion 

This paper has reviewed the school philosophy and policies of students such as 
these nationally, finding that most students attended schools in which the primary focus 
for learning-handicapped students was to teach academic skills. Most also attended 
schools that graded the work of mainstreamed students (but not of students in special 
classes) according to the same standards as the work of non-special education 
students, and where special education students took the same version of minimum 
competency tests and were held to the same standards for passing the tests as other 
students. 

Although most schools stated that they made counseling/therapy, life skills/ 
occupational therapy, social work, and human aides or tutors available to special 
education students who needed them, almost half of students had received no services 
at all in the preceding year. Fewer than half of students had received 
counseling/therapy from any source, and only 1/3 had received such services from their 
schools. Fifteen percent had received tutoring services, and 19% had received life 
skills training/occupational therapy. Twenty-nine percent of students who spent some 
time in regular education classes had received no services and were in schools that did 
not reduce class sizes or use human aides in classes where special education students 
were mainstreamed. 

The secondary school outcomes of students classified as SED leave much room for 
improvement. With an average of 17 days absent in a year, their rate of absenteeism 
was the highest of youth with any type of disability (Wagner, 1991). On average, their 
grades were low; almost half of students who received grades had failed one or more 
courses in the preceding year, and 16% were retained in grade at the end of the school 
year. The culmination of this discouraging set of outcomes is that as many youth left 
secondary school by dropping out or being suspended/expelled as left by graduating. 

What makes a difference in secondary school performance and outcomes? 
Unfortunately, because of small sample sizes, relationships of variables with mode of 
school completion could not be examined. Multivariate models were able to explain 
very little of the variation in absenteeism and course failure. We have seen that none 
of the individual characteristics usually associated with secondary school performance 
and outcomes for other youth— gender, ethnic background, coming from a low-SES 
family, or IQ — showed significant associations with either absenteeism or course 
failure. In contrast, the models estimated suggest that tutoring and personal 
counseling/therapy help improve student outcomes. 

Given the nature of the disabilities included in the label "seriously emotionally 
distrubed," these findings are hardly surprising. Consider how difficult life, let alone 
schoolwork, must be for these youth, who struggle with disabilities that can include 
constant shifts in their reality caused by delusions and hallucinations, or depressions so 
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deep they can have difficulty even moving, or extreme anxiety caused by abusive 
parents. With such disabilities, it is hardly surprising that being male or female, white, 
black, or Hispanic make no difference. On the other hand, it is also not surprising that 
services intended to help youth cope, with both schoolwork and with greater problems, 
do make a difference. 

Very little of the variance in school performance is explained by the models. Among 
the possible reasons for this lack of explanatory power are the absence of measures of 
severity of disability and quality of programs and services. Yet examining phenomena 
such as severity and quality is difficult in large-scale quantitative research such as the 
NLTS. Furthermore, other, less understood, factors also may be operating. Given the 
poor outcomes of youth classified as seriously emotionally disturbed, further research is 
crucial, as has been recognized in PL 101-476. Yet, at this juncture, it seems that 
research in a more qualitative vein will be most helpful in illuminating what can make a 
difference in the outcomes of students classified as seriously emotionally disturbed. 
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Appendix A 

BACKGROUND INFORMATION ON THE NLTS SAMPLE 



This appendix provides somewhat greater detail on several methodological aspects of the 
NLTS, including: 

• Data collection components. 

• Sampling of districts, schools, and students. 

• Weighting of NLTS data. 

• Estimation and use of standard errors. 

• Construction of comparison groups from the general population using the National 
Longitudinal Survey of Youth (U.S. Department of Labor). 

Components of the NLTS 

The NLTS has several components: 

• The Parent/Youth Survey. In the summer and fall of 1987, parents were interviewed by 
telephone to determine information on family background and expectations for the youth in 
the sample, characteristics of the youth, experiences with special services, the youths 1 
educational attainments (including postsecondary education), employment experiences, 
and measures of social integration. Parents rather than youth were selected as 
respondents for the first wave of data collection because of the need for family background 
information and because, with most students still being in secondary school and living at 
home, parents were believed to be accurate respondents for the issues addressed. The 
survey was repeated in 1990, when youth were interviewed if they were able to respond. 

• School Records. In 1 987 information was abstracted from students' school records for 
the most recent year in secondary school (either the 1985-86 or 1986-87 school year). 
This information related to courses taken, grades received (if in a graded program), 
placement, related services received from the school, status at the end of the year, 
attendance, IQ, and experiences with minimum competency testing. School transcripts 
were collected in 1990 for youth who had been in secondary school at any time since the 
1986-87 school year. 

• School Program Survey. In 1 987, schools attended by sample students in the 1 986-87 
school year were surveyed for information on enrollment, staffing, programs and related 
services offered to secondary special education students, policies affecting special 
education programs and students, and community resources for the disabled. 

• Student School Program Purvey. In 1990, this survey obtained information abou: 
youth who still were in secondary school. Respondents were teachers familiar with 
students 1 school programs. They reported about students 1 in-class performance, class 
size, school climate, and transition planning activities that had occurred for each student. 

• Explanatory Substudles. Studies involving subsamples of youth in selected disability 
categories examined in greater depth students 1 secondary school programs, the patterns 
of transition outcomes achieved by youth who were out of secondary school, and the 
relationship between school experiences and outcomes. Data were collected for in- 
school youth in 1988 and 1989 and for out-of-school youth in 1989. 
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The NLTS Sample 

The initial NLTS sample was constructed in two stages. A sample of 450 school districts 
was selected randomly from the universe of approximately 14,000 school districts serving 
secondary (grade 7 or above) students in special education,* which had been stratified by 
region of the country, a measure of district wealth involving the proportion of students in poverty 
(Orshansky percentile), and student enrollment. Because not enough districts agreed to 
participate, a replacement sample of 178 additional districts was selected. More than 80 state- 
supported special schools serving secondary-age deaf, blind, and deaf-blind students also were 
invited to participate in the study. A total of 303 school districts and 22 special schools agreed 
to have their students selected for the study. 

Analysis of the potential bias of the district sample indicated virtually no systematic bias that 
would have an impact on study results when participating districts were compared to 
nonparticipants on several characteristics of the students served, participation in Vocational 
Rehabilitation programs, the extent of school-based and community resources for the disabled, 
the configuration of other education agencies serving district students, and metropolitan status 
(see Javitz, 1990 for more information on the LEA sample). The one exception was a significant 
underrepresentation of districts serving grades kindergarten through eight. Many of these 
districts did not consider themselves as secondary school districts, even though they served 
grades seven and eight, which are considered secondary grade levels. In addition, bias may 
exist on factors for which data were not available for such comparisons. 

Students were selected from rosters compiled by districts, which were instructed to include 
all students in special education in the 1985-86 school year who were in grades 7 through 12 or 
whose birthdays were in 1972 or before, whether or not they were served within the district or 
outside the district (e.g., in state-supported residential schools). Rosters were stratified into 3 
age groups (13 to 15, 16 to 18, over 18) for each of the 1 1 federal special education disability 
categories and youth were randomly selected from each age/disability group so that 
approximately 800 to 1 ,000 students were selected in each disability category (with the 
exception of deaf-blind, for which fewer than 1 00 students were served in the districts and 
schools included in the sample). 

In part because of the time lapse between sample selection and data collection, many 
students could not be located at the addresses or telephone numbers provided by the schools. 
Of the 12,833 students selected for the sample, about one-third could not be reached by 
telephone for the 1987 parent interview. (For more than half of these, addresses and telephone 



The 1983 Quality Education Data, Inc. (QED) database was used to construct the sampling frame. QED is a 
private nonprofit firm located in Denver, Colorado. Special education cooperatives and other special service units 
were not sampled directly (83% of special education students are served directly by school districts; Moore et al., 
1988). However, instructions to districts for compiling student rosters asked districts to include on their listing any 
students sent from their district to such cooperatives or special service units. Despite these instructions, some 
districts may have underreported students served outside the district. 



numbers were not provided by the schools/districts from which they were sampled.) This 
relatively high rate of inability to reach sample members confirmed the importance of including 
in the NLTS a substudy of nonrespondents to determine whether those who were reached for 
the telephone interview were a representative sample of the population to which the study was 
intended to generalize. To identify whether bias existed in the interview sample, interviewers 
went to 28 school districts with relatively high nonresponse rates to locate and interview in 
person those who could not be reached by telephone. Of the 554 sought for in-person 
interviews, 442 were found and interviewed, a response rate of 80%. A comparison of 
telephone interview respondents with in-person interview respondents showed that the 
telephone sample underrepresented lower-income households. The sample was reweighted to 
adjust for that bias, as described in the next section. 

Data from 1990 on trends in postschool outcomes are based on the responses of 1,990 
youth who satisfied four conditions: 1) they were enrolled in special education at a secondary 
school in the 1 985-86 school year, 2) they left secondary school by September 1987, 3) their 
parent or guardian completed an interview in the wave 1 data collection effort, and 4) either the 
parent or youth completed a telephone interview or mail questionnaire in the wave 2 data 
collection effort. These youth were weighted to represent all youth enrolled in special education 
in the 1985-86 school year who had left secondary school by September 1987. 

Weighting Procedures and the Population to Which Data Generalize 

Youth with disabilities for whom data could be gathered were weighted to represent the U.S. 
population of students in special education in the 1985-86 school year who were in grades 7 
through 1 2 or at least 13 years old. Because it is a sample of students at various ages, the 
NLTS sample does not generalize to youth who had dropped out of school before that age. For 
example, the sample of 18-year-olds generalizes to youth who were 18 and still in secondary 
school in 1985-86, not to all 18-year-olds with disabilities, many of whom may had left school at 
an earlier age. 

In performing sample weighting for wave 1 (1987), three mutually exclusive groups of 
sample members were distinguished: 

(A) Youth whose parents responded to the telephone interview. 

(B) Youth whose parents did not respond to the telephone interview but were 
interviewed in person. 

(C) Youth whose parents did not respond to either the telephone or in-person 
interviews but for whom we obtained a record abstract. 



A major concern in weighting was to determine whether there was a nonresponse bias and 
to calculate the weights in such a way as to minimize that bias. There was a potential for three 
types of nonresponse bias:* 

(1 ) Bias attributable to the inability to locate respondents because they had moved or 
had nonworking telephone numbers. 

(2) Bias attributable to refusal to complete an interview (only 3% of those available to 
be interviewed refused). 

(3) Bias attributable to circumstances that made it infeasible to locate or process a 
student's school record. 

Of these three types of nonresponse, the first was believed to be the most frequent and to have 
the greatest influence on the analysis. Type 1 bias also was the only type of nonresponse that 
could be estimated and corrected. 

The magnitude of type 1 nonresponse bias was estimated by comparing responses to items 
available for the three groups of respondents (after adjusting for differences in the frequency 
with which youth in different disability categories were selected and differences in the size of the 
LEAs selected). Group A was wealthier, more highly educated, and less likely to be minority 
than group B. !n addition, group A was more likely to have students who graduated from high 
school than groups B or C (which had similar dropout rates). Groups A and B were compared 
on several additional measures for which data were unavailable for group C. The youth 
described by the two groups were similar on these additional items, including gender, 
employment status, pay, functional skills, association with a social group, and length of time 
since leaving school. Adjusting sample weights to eliminate bias in the income distribution 
eliminated bias in parental educational attainment and ethnic composition, but did not affect 
differences in dropout rates. Groups B and C were large enough that if they were treated the 
same as group A in the weighting process, the resulting dropout distribution would be 
approximately correct. 

Sample weighting involved the following steps: 

• Data from the first groups of sample members were used to estimate the income 
distribution for each disability category that would have been obtained in the absence 
of type 1 nonresponse bias. 

• Respondents from all three groups were combined and weighted up to the universe 
by disability category. Weights were computed within strata used to select the 
sample (i.e., LEA size and wealth, student disability category and age). 

• Weights from three low-incidence disability categories (deaf, orthopedically impaired, 
and visually impaired) were adjusted to increase the effective sample size. These 



* We assumed that nonrespondents who could not be located because LEAs did not provide student names would 
have chosen to participate at about the same rate as parents in districts in which youth could be identified. The 
remaining nonrespondents would presumably have been distributed between the three types of nonresponse 
mentioned above. 
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adjustments consisted primarily of slightly increasing the weights of students in larger 
LEAs and decreasing the weights of students in smaller LEAs. Responses before 
and after these weighting adjustments were nearly identical. In addition, the three 
deaf/blind youth from medium-size or smaller districts, who had large weights, were 
removed from the sample to increase the effective sample size. Thus, NLTS results 
do not represent the very small number of deaf/blind students in medium-size or 
smaller LEAs. 

• The resulting weights were adjusted so that each disability category exhibited the 
appropriate income distribution estimated in step 1 above. These adjustments were 
modest (relative to the range of weights within disability category); the weights of the 
poorest respondents were multiplied by a factor of approximately 1 .6 and the weights 
of the wealthiest respondents were multiplied by a factor of approximately .7. 

Because analyses of postschool outcomes included 1 990 data for only a subset of youth, 
new weights were needed for 1990 data. The first step in weighting the 1,990 out-of-school 
youth was to identify a group of 3,046 youth who had been enrolled in special education in the 
1985-86 school year, who had left secondary school by September 1987, and for whom we had 
sufficient data so that these youth had been given a weight in the wave 1 analysis. (This did not 
require that the parent of the youth complete a parent/guardian interview; having a school 
record abstract was sufficient to receive a wave 1 weight.) Use of this wave 1 weight allowed 
the results for these 3,046 youth to be projected to the corresponding national population (that 
is, youth who were enrolled in special education in secondary school in 1985-86 and who had 
left secondary school by September 1987). 

The second step in weighting was to use the group of 3,046 youth and their wave 1 weights 
to calculate distributions of the following: 

• Age— The primary categories were 15 to 17 years, individual years of age from 18 to 22, 
and a combined category of 23 and above. 

• Ethnic background— The primary categories were black; white; Hispanic; and a 
combined category for Indian/Alaskan, Asian/Pacific Islander; and other. In addition 
there was a category for "don't know** or refusals, and a category for missing (typically 
because the data collection instrument that was completed for youth did not ask for this 
information). 

• School completion status— The primary categories were graduated, aged out, and a 
combined category of dropped out, suspended, or expelled. In addition there was a 
category for **don*t know** or "plans to return to school.** 

• Gender. 

• Household Income In 1986 (or 1 990 if 1 986 data was not available). The primary 
categories were under $12,000; $12,000 to $19,999; $20,000 to $24,999; under $25,000 
but otherwise unspecified; $25,000 to $37,999; $38,000 to $50,000; and over $50,000. 
Those with incomes of $25,000 or over but otherwise unspecified were grouped with 
those with household incomes between $25,000 and $37,999. In addition there was a 
category for those with missing information and a category for those who responded 
"don't know,** refused to answer, or indicated that the youth was institutionalized. 
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The third step was the use of a weighting program to calculate weights for the 1 ,990 youth 
so that they matched the demographic distributions of the 3,046 youth. The weighting was 
accomplished using Deming's algorithm, which iteratively modified the wave 1 weights for the 
1 ,990 youth until they generated demographic marginals that were very similar to those 
obtained using the 3,046 youth. Each disability class was weighted separately and in general 
the demographic marginals were matched within a fraction of 1 percent. (Only for the 
deaf/blind, where sample sizes were very small, did any marginals fail to match within 1 percent, 
and here they differed no more than 2%.) 

Estimation of Standard Errors 

The NLTS stratified cluster sample introduces design effects that reduce the precision of 
estimates for a sample of a given size, compared with a simple random sample. The design 
effects within the NLTS affect the precision of estimates to varying degrees for different 
subpopulations and different variables. Pseudo-replication is widely accepted as a variance 
estimation technique in the presence of design effects. However, it is not cost-effective for 
estimating the standard errors of the thousands of variables and subpopulations tabulated in the 
numerous NLTS reports and its statistical almanacs. Therefore, pseudo-replication was 
conducted on a limited number of variables to calibrate a cost-effective approximation formula, 
using the following procedures: 

• A set of 25 variables representing the parent interview, school program survey, and 
record abstract was identified for the purpose of developing a statistical 
approximation formula; these included 16 nominal variables and 9 continuous 
variables. 

• Standard errors of the weighted means of the selected variables were estimated in 
two ways. The first procedure involved pseudo- replication. For each variable, 
standard errors were calculated for students in each disability category and for the 
total sample (300 standard errors) using a partially balanced experimental design 
specifying how youth were to be allocated to 16 half-samples. The sample was split 
on the basis of the school districts and special schools from which youth originally 
were sampled. Districts and schools were paired on the basis of enrollment and a 
measure of poverty, and one member of each pair was assigned to each half- 
sample. Sample weights were computed for each half-sample as if those in the half- 
sample were the only study participants. 

The following formula was used to estimate the standard error of the mean for youth in 
all conditions: 



Standard error = [(1/16) I (M r M) 2 ] 1 ' 2 

i 

where Mj is the mean calculated for youth in one of the 16 half- samples), M is the mean 
response calculated from the full sample, and the summation extends over all 16 half- 
samples. (Note that responses to questions from the school program survey were 
attached to the records of students in the responding schools so that means for these 
items were computed using student weights.) 
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The second estimation procedure involved an approximation formula based on an 
estimate of the effective sample size for each disability category and the total sample. 
The sampling efficiency (E) for a group was calculated using the following formula: 



E = Mw 2 /(M w 2 + S w 2) 

where and S w are the mean and standard deviation of the student weights over all 
members of the group. The approximation formula for the standard error of the weighted 
mean of nominal variables is: 

Standard error « [P(1-P)/(N x E)] 1/2 

where P is the full-sample weighted proportion of "yes" responses to a particular 
question in the group, N is the unweighted number of yes 11 or "no" responses to the 
question in the group, and E is the sampling efficiency of the group. The approximation 
formula for the standard error of the mean of a continuous variable is: 

Standard error = [S 2 /(N x E)p 

where S 2 is the variance of responses in the group for the continuous variable 
(computed with frequencies equal to full-sample weights) and N is the unweighted 
number of respondents to the question in the group. These formulas were used to 
compute a total of 300 standard errors for the same variables and groups addressed 
using pseudo-replication. 

To assess the accuracy of the standard errors produced by these formulas, we used 
scatter plots to compare them with standard errors produced using pseudo-replication. 
For both nominal and continuous variables, the approximate best fit was a 45 degree 
line. That is, on average, the formula based on estimates of effective sample size 
neither systematically overestimated nor underestimated the standard error obtained 
using pseudo-replication, arguing for use of the more cost-effective estimation formulas. 
However, because error remains in the estimates that might result in underestimating 
the true standard errors in some instances, we took a conservative approach and 
multiplied the standard errors produced using the estimation formulas by 1 .25. The vast 
majority of the standard errors so obtained were larger than the standard errors obtained 
by pseudo-replication. Thus, standard errors were calculated using the effective sample 
size estimation formulas and increased by a factor of 1 .25. 



Creating Comparison Groups from the General Population of Youth 

We have created two comparison groups frcni the general population of youth to use as 
benchmarks against which to interpret outcomes of youth with disabilities. The first group is a 
sample of youth from the general population, based on data from the National Longitudinal 
Survey of Youth (NLSY, U.S. Department of Labor). This group permits us to identify 
differences between youth with disabilities and the general population. Hovyever, we cannot 
attribute those differences to the presence of a disability because Chapter 2 has illustrated that 
youth with disabilities differed from youth in the general population on demographic 
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characteristics that would be expected to influence their outcomes (e.g., gender, ethnicity). 
Hence, a second comparison group was constructed from the NLSY that has the same 
distribution as youth with disabilities on important demographic variables. The construction of 
these two groups is described below. 

The NLSY contains data for more than 12,000 noninstitutionalized youth who were between 
the ages of 13 and 21 in 1979. These youth have been interviewed annually from 1979 to the 
present concerning a wide variety of topics, including their family background, schooling, 
employment, marital status, and living arrangements. For the present study, data from the 
1979-1983 interviews were used; after those years, youth in the NLSY were generally older than 
youth in the NLTS. 

Because the universe of the NLTS is youth who were in special education programs in 
1985-86, while the universe for the NLSY is all youth (regardless of present or past school 
status), the following steps were taken to achieve comparability. First, only NLSY youth who 
were currently in school or had been in school during the current or previous academic year 
were included in the analysis. Second, comparisons were restricted to youth between 15 and 20 
years of age. This was done primarily because very few NLSY youth over age 20 met the 
requirement of having been in secondary school the academic year before the interview. Little 
is lost by this restriction because the NLTS sample contains very few individuals below the age 
of 15 and relatively few over age 20. 

Thus, we used all the in-school observations and any observations when a person was out 
of school, but had been in school during the academic year before the interview. There were up 
to 5 in-school interviews for a given youth. For most people, only one out-of-school observation 
was included. Two out-of-school interviews could occur if a youth left school during an 
academic year but before the spring interview. In that case, the interviews of the spring of that 
academic year and the next spring were included. 

NLSY provides sampling weights based on respondents' probability of selection. However, 
our use of multiple observations per respondent for many analyses resulted in older youth being 
overrepresented. We corrected this bias by multiplying each individual's weight by: 

Weighted N of individuals of the youth's age in 1980 
Weighted N of the youth's age for all observations in the sample. 

For analyses that used multiple observations, this weight was used. For analyses that used 
one observation only (for instance, data on arrests came only from the 1980 interview), the 
original weight supplied by the NLSY was used. 

As indicated above, youth with disabilities differ in several demographic characteristics from 
the general population of youth. The comparison group we constructed to "hold constant" these 
differences was formed by weighting the NLSY data to match the distribution of selected 
demographic characteristics of youth with disabilities. Using these weights, the comparison 




population has the same distributions of gender, ethnicity, and head of household's education 
as the population of youth with disabilities. 

Despite our adjustments, some important noncomparabilities remain. They are as follows: 

• Respondent NLTS interviewed parents, while NLSY interviewed youth. Although 
there is some evidence that parents in the general population tend to underreport the 
employment activities of their teenage children (Freeman and Medoff, 1982), the 
extent to which parents and youth differ in reporting other phenomena is not known. 

• Month of Interview. The modal month of interview was August for the NLTS and 
March for the NLSY. The two outcomes most affected by differences in timing of 
interview are school completion status and employment status. Fortunately, NLSY 
data included youths' employment status as of August 15, and we were able to 
construct a variable on school completion status as of the summer after the 
interview. However, most data on occupational distributions, part-time/full-time 
status, and wages come from the summer for NLTS youth and the spring for NLSY 
youth. 

Year of Interview. NLTS interviews took place in 1987, while NLSY data come from 
1979-1982. Readers should be sensitive to the fact that period effects may have 
influenced some variables. We adjusted for period effects for only one variable, 
wages, by operationalizing wages as the percent of the population earning the 
minimum wage or less. 

• Time out of school. The most important consequence of differences in the month 
of interview affect analyses of data for youth who were no longer in secondary 
school. More than three-fourths (76%) of NLSY secondary school graduates in the 
sample (weighted) had been out of school between 9 and 1 1 months when they were 
interviewed. In contrast, about 56% of NLTS graduates had been out of school 
about 2 months, and about 44% had been out of school about 14 months. 

• Unmeasured or uncontrolled demographic differences. The groups may 
continue to differ in unmeasured ways or in ways that were not adjusted for in the 
reweighting. For example, we were not able to weight the comparison population by 
urbanicity, despite knowing that NLTS and NLSY samples differ significantly on this 
factor, because of noncomparability of the measures of urbanicily in the two data 
sets. 

• Exact wording of questions and response categories. Wording of questions and 
response categories differed between the NLTS and the NLSY. Considerable 
research has shown responses to items can be affected by these types of 
differences (e.g., Hippler, Schwarz, and Sudman, 1987). 
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Reports and Papers 
Based on the NLTS 



Papers available: 

• "What Makes a Difference? Factors Related to Postsecondary School Attendance for Young People 
with Disabilities." L Newman and R. Cameto, 1993. 48 pp. $10.00. [Order No. 168] 

• "Human Capital and Structural Explanations of Post-School Success for Youth with Disabilities: A 
Latent Variable Exploration of the National Longitudinal Transition Study." J. Blackorby, S. Siegel, and G. 
Hancock, 1993. 44 pp. $10.00. [Order No. 169] 

• "What Happens Next? Trends in Postschool Outcomes of Youth with Disabilities." M.Wagner, 
R. D'Amico, C. Marder, L.Newman, and J. Blackorby, 1992. 328 pp. $32.00. [Order No. 166] 

• "Youth Classified as Seriously Emotionally Disturbed: How Well Are They Being Served?" C. Marder, 
1992. 25 pp. $10.00. [Order No. 158] 

• "Hispanic Secondary School Students with Disabilities: How Are They Doing?" L. Newman, 1992. 
35 pp. $10.00. [Order No. 157] 

• "Being Female— A Secondary Disability? Gender Differences in the Transition Experiences of Young 
People with Disabilities." M.Wagner, 1992. 50 pp. $10.00. [Order No. 156] 

• 'The Early Work Experiences of Youth with Disabilities: Trends in Employment Rates and Job 
Characteristics." R. D'Amico and C. Marder, 1991. 56 pp. $15.00. [Order No. 147] 

• "Dropouts with Disabilities: What Do We Know? What Can We Do?" M.Wagner, 1991. 80 pp. 
$15.00. [Order No. 146] 

• "How Well Are Youth with Disabilities Really Doing? A Comparison of Youth with Disabilities and Youth 
in General." C. Marder, March 1992. 92 pp. $15.00. [Order No. 144] 

• "Youth with Disabilities: How Are They Doing? The First Comprehensive Report from the National 
Longitudinal Transition Study of Special Education Students." M. Wagner et al. 1991 . 600 pp. 
$40.00. [Order No. 135] 

• "Parents' Reports of Students' Involvement with Vocational Rehabilitation Agencies in the First Years 
After Secondary School." M. Wagner and R. Cox, 1991. 50 pp. $12.00. [Order No. 134] 

• 'The Relationship Between Social Activities and School Performance for Secondary Students with 
Learning Disabilities." L. Newman, 1991. 52 pp. $10.00. [Order No. 133] 

• 'The Benefits Associated with Secondary Vocational Education for Young People with Disabilities." 
M.Wagner, 1991. 66 pp. $10.00. [Order No. 132] 

• 'The National Longitudinal Transition Study of Special Education Students: Report on Sample Design 
and Limitations, Wave 1 (1987)." H. Javitz and M. Wagner, 1990. 71 pp. $18.00. [Order No. 131] 

• M The National Longitudinal Transition Study of Special Education Students: Report on Procedures for 
the First Wave of Data Collection (1987)." M. Wagner, L. Newman, and D. Shaver, 1989 (includes data 
collection instruments). 280 pp. $25.00. [Order No. 126] 
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• "The National Longitudinal Transition Study of Special Education Students: Report on Procedures for 
the Second Wave of Data Collection (1990)." C. Marder, K. Habina, and N. Prince, 1992 (includes data 
collection instruments). 220 pp. $20.00. [Order No. 165] 

• "The School Programs and School Performance of Secondary Students Classified as Learning 
Disabled: Findings from the National Longitudinal Transition Study of Special Education Students." 
M. Wagner, 1990. 27 pp. $10.00 [Order No. 125] 

• 'The Transition Experiences of Youth with Disabilities: A Report from the National Longitudinal 
Transition Study." M. Wagner, 1989. 42 pp. $10.00. [Order No. 118] 

• "Making the Transition: An Explanatory Model of Special Education Students' Participation in 
Postsecondary Education." P. Butler-Nalin, C. Marder, and D. Shaver, 1989. 41 pp. $10.00. 
[Order No. 117] 

• "Educational Programs and Achievements of Secondary Special Education Students: Findings from 
the National Longitudinal Transition Study." M. Wagner and D. Shaver, 1989. 41 pp. $10.00. 
[Order No. 116] 

• "Dropouts: The Relationship of Student Characteristics, Behaviors, and Performance for Special 
Education Students." P. Butler-Nalin and C. Padilla, 1989. 41 pp. $10.00. [Order No. 115] 



Publications Order Form 



Order 
No. 


Title 


Unit Price 


Quantity 


Total 
Price 



















































































Subtotal: 



If payment not included, $10.00 service charge for invoicing: 

(In CA add appropriate state and local sales tax) Tax: 

Shipping and handling are included. If possible, enclose payment with order. 

If payment is not e nclosed, there will be a charge of $10.00. Total^ 

Ship to: Name 

Affiliation 

Address 



City, State, ZIP . 

Mail order form and payment (payable to the order of SRI International) to: 
National Longitudinal Transition Study, Room BS178 
SRI International 
333 Ravenswood Avenue 
Menlo Park CA 94025 
(415)859-5109 - 



t 



Appendix C 

CHARACTERISTICS OF THE SAMPLE USED IN EXPLANATORY MODELS 



Models of 
Absenteeism and 
Course Failure 



Percent/ 
Mean 

Student Characteristics 
Gender 

Male 80.1 

Female 19.1 

Ethnic background 

White 79.5 

Black 16.1 

Hispanic 3.6 

Head of household of origin 

has less than 12 years' education 45.0 

Single-parent household of origir. 38.3 

Mean IQ score 86.8 

Services in preceding 12 months 

Counseling/therapy 49.7 

Tutor or reader 17.5 

Life skills/occupational therapy 23.9 

School philosophy and practices 

Primary function of school 

Teach academic skills 51 .9 

Teach independent living skills 37.6 

Train for employment 1 .4 

Combination or other 9. 1 

Mainstreamed students expected to 

keep up with class 38.3 

Mean percentage of time in regular 

classes 57.2 

N 190 



Standard 
Error 



(4.2) 
(4.2) 



(4.4) 
(4.0) 
(2.0) 



(5.6) 
(5.5) 
(1.4) 



(5.2) 
(4.0) 
(4.5) 



(5.2) 
(5.1) 
(1.2) 
(3.1) 



(5.1) 



(1.4) 



Model of 
Dropping Out 

Percent/ Standard 
Mean Error 



79.4 
20.6 



78.8 
15.8 
4.5 



47.5 
38.4 
86.6 



48.4 
17.3 
21.3 



52.0 
36.3 
1.1 
10.6 



38.8 

57.7 
156 



(4.7) 
(4.7) 



(4.5) 
(5.0) 
(2.5) 



(6.8) 
(6.1) 
(1.5) 



(5.8) 
(4.4) 
(4.8) 



(5.8) 
(5.6) 
(1.2) 
(3.8) 



(5.7) 



(3.7) 
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